{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jWESX0tpdrE-"
      },
      "source": [
        "##### Copyright 2024 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "cellView": "form",
        "id": "YQvTrJpxzRlJ"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3hp_P0cDzTWp"
      },
      "source": [
        "# Gemini 2.0 - Multimodal live API: Tool use"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OLW8VU78zZOc"
      },
      "source": [
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/panaversity/learn-agentic-ai/blob/main/12_langchain_ecosystem/langgraph/langgraph_easy_tutorial/03_gemini_2_live_api_tools/live_api_tool_use.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y7f4kFby0E6j"
      },
      "source": [
        "This notebook provides examples of how to use tools with the multimodal live API with [Gemini 2.0](https://ai.google.dev/gemini-api/docs/models/gemini-v2).\n",
        "\n",
        "The API provides Google Search, Code Execution and Function Calling tools. The earlier Gemini models supported versions of these tools. The biggest change with Gemini 2 (in the Live API) is that, basically, all the tools are handled by Code Execution. With that change, you can use **multiple tools** in a single API call, and the model can use multiple tools in a single code execution block.  \n",
        "\n",
        "This tutorial assumes you are familiar with the Live API, as described in the [this tutorial](https://github.com/google-gemini/cookbook/blob/main/gemini-2/live_api_starter.ipynb)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Mfk6YY3G5kqp"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d5027929de8f"
      },
      "source": [
        "### Install SDK\n",
        "\n",
        "The new **[Google Gen AI SDK](https://ai.google.dev/gemini-api/docs/sdks)** provides programmatic access to Gemini 2.0 (and previous models) using both the [Google AI for Developers](https://ai.google.dev/gemini-api/docs) and [Vertex AI](https://cloud.google.com/vertex-ai/generative-ai/docs/overview) APIs. With a few exceptions, code that runs on one platform will run on both. This means that you can prototype an application using the Developer API and then migrate the application to Vertex AI without rewriting your code.\n",
        "\n",
        "More details about this new SDK on the [documentation](https://ai.google.dev/gemini-api/docs/sdks) or in the [Getting started](../gemini-2/get_started.ipynb) notebook."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "46zEFO2a9FFd"
      },
      "outputs": [],
      "source": [
        "!pip install -U -q google-genai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CTIfnvCn9HvH"
      },
      "source": [
        "### Setup your API key\n",
        "\n",
        "To run the following cell, your API key must be stored it in a Colab Secret named `GOOGLE_API_KEY`. If you don't already have an API key, or you're not sure how to create a Colab Secret, see [Authentication](../quickstarts/Authentication.ipynb) for an example."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "A1pkoyZb9Jm3"
      },
      "outputs": [],
      "source": [
        "from google.colab import userdata\n",
        "import os\n",
        "\n",
        "os.environ['GOOGLE_API_KEY']=userdata.get('GOOGLE_API_KEY')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y13XaCvLY136"
      },
      "source": [
        "### Initialize SDK client\n",
        "\n",
        "The client will pickup your API key from the environment variable.\n",
        "To use the live API you need to set the client version to `v1alpha`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "HghvVpbU0Uap"
      },
      "outputs": [],
      "source": [
        "from google import genai\n",
        "\n",
        "client = genai.Client(http_options= {\n",
        "      'api_version': 'v1alpha'\n",
        "})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QOov6dpG99rY"
      },
      "source": [
        "### Select a model\n",
        "\n",
        "Multimodal Live API are a new capability introduced with the [Gemini 2.0](https://ai.google.dev/gemini-api/docs/models/gemini-v2) model. It won't work with previous generation models."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "27Fikag0xSaB"
      },
      "outputs": [],
      "source": [
        "model_name = \"gemini-2.0-flash-exp\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pLU9brx6p5YS"
      },
      "source": [
        "### Imports"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "yMG4iLu5ZLgc"
      },
      "outputs": [],
      "source": [
        "import asyncio\n",
        "import contextlib\n",
        "import json\n",
        "import wave\n",
        "\n",
        "from IPython import display\n",
        "\n",
        "from google import genai\n",
        "from google.genai import types"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yrb4aX5KqKKX"
      },
      "source": [
        "### Utilities"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rmfQ-NvFI7Ct"
      },
      "source": [
        "You're going to use the Live API's audio output, the easiest way hear it in Colab is to write the `PCM` data out as a `WAV` file:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "p2aGpzlR-60Q"
      },
      "outputs": [],
      "source": [
        "@contextlib.contextmanager\n",
        "def wave_file(filename, channels=1, rate=24000, sample_width=2):\n",
        "    with wave.open(filename, \"wb\") as wf:\n",
        "        wf.setnchannels(channels)\n",
        "        wf.setsampwidth(sample_width)\n",
        "        wf.setframerate(rate)\n",
        "        yield wf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KfdD9mVxqatm"
      },
      "source": [
        "Use a logger so it's easier to switch on/off debugging messages."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "wgHJgpV9Zw4E"
      },
      "outputs": [],
      "source": [
        "import logging\n",
        "logger = logging.getLogger('Live')\n",
        "#logger.setLevel('DEBUG')  # Switch between \"INFO\" and \"DEBUG\" to toggle debug messages.\n",
        "logger.setLevel('INFO')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4hiaxgUCZSYJ"
      },
      "source": [
        "## Get started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LQoca-W7ri0y"
      },
      "source": [
        "Most of the Live API setup will be similar to the [starter tutorial](../gemini-2/live_api_starter.ipynb). Since this tutorial doesn't focus on the realtime interactivity of the API, the code has been simplified: This code uses the Live API, but it only sends a single text prompt, and listens for a single turn of replies."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "lwLZrmW5zR_P"
      },
      "outputs": [],
      "source": [
        "n = 0\n",
        "async def run(prompt, modality='TEXT', tools=None):\n",
        "  global n\n",
        "  if tools is None:\n",
        "    tools=[]\n",
        "\n",
        "  config = {\n",
        "          \"tools\": tools,\n",
        "          \"generation_config\": {\n",
        "              \"response_modalities\": [modality]}}\n",
        "\n",
        "  async with client.aio.live.connect(model=model_name, config=config) as session:\n",
        "    display.display(display.Markdown(prompt))\n",
        "    display.display(display.Markdown('-------------------------------'))\n",
        "    await session.send(prompt, end_of_turn=True)\n",
        "\n",
        "    audio = False\n",
        "    filename = f'audio_{n}.wav'\n",
        "    with wave_file(filename) as wf:\n",
        "      async for response in session.receive():\n",
        "        logger.debug(str(response))\n",
        "\n",
        "        server_content = response.server_content\n",
        "        if server_content is not None:\n",
        "          a = handle_server_content(wf, server_content)\n",
        "          audio = audio or a\n",
        "\n",
        "        tool_call = response.tool_call\n",
        "        if tool_call is not None:\n",
        "          await handle_tool_call(session, tool_call)\n",
        "\n",
        "\n",
        "  if audio:\n",
        "    display.display(display.Audio(filename, autoplay=True))\n",
        "    n = n+1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ngrvxzrf0ERR"
      },
      "source": [
        "Since this tutorial demonstrates several tools, you'll need more code to handle the different types of objects it returns.\n",
        "\n",
        "- The `code_execution` tool can return `executable_code` and `code_execution_result` parts.\n",
        "- The `google_search` tool may attach a `grounding_metadata` object."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "CypjqSb-0C-Q"
      },
      "outputs": [],
      "source": [
        "def handle_server_content(wf, server_content):\n",
        "  audio = False\n",
        "  model_turn = server_content.model_turn\n",
        "  if model_turn:\n",
        "    for part in model_turn.parts:\n",
        "      text = part.text\n",
        "      if text is not None:\n",
        "        display.display(display.Markdown(text))\n",
        "\n",
        "      inline_data = part.inline_data\n",
        "      if inline_data is not None:\n",
        "        print('.', end='')\n",
        "        pcm_data = inline_data.data\n",
        "        wf.writeframes(pcm_data)\n",
        "        audio = True\n",
        "\n",
        "      executable_code = part.executable_code\n",
        "      if executable_code is not None:\n",
        "        display.display(display.Markdown('-------------------------------'))\n",
        "        display.display(display.Markdown(f'``` python\\n{executable_code.code}\\n```'))\n",
        "        display.display(display.Markdown('-------------------------------'))\n",
        "\n",
        "      code_execution_result = part.code_execution_result\n",
        "      if code_execution_result is not None:\n",
        "        display.display(display.Markdown('-------------------------------'))\n",
        "        display.display(display.Markdown(f'```\\n{code_execution_result.output}\\n```'))\n",
        "        display.display(display.Markdown('-------------------------------'))\n",
        "\n",
        "  grounding_metadata = getattr(server_content, 'grounding_metadata', None)\n",
        "  if grounding_metadata is not None:\n",
        "    display.display(\n",
        "        display.HTML(grounding_metadata.search_entry_point.rendered_content))\n",
        "\n",
        "  return audio"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dPnXSNZ5rydM"
      },
      "source": [
        "- Finally, with the `function_declarations` tool, the API may return `tool_call` objects. To keep this code minimal, the `tool_call` handler just replies to every function call with a response of `\"ok\"`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "EmTKF_DtrY4U"
      },
      "outputs": [],
      "source": [
        "async def handle_tool_call(session, tool_call):\n",
        "  for fc in tool_call.function_calls:\n",
        "    tool_response = types.LiveClientToolResponse(\n",
        "        function_responses=[types.FunctionResponse(\n",
        "            name=fc.name,\n",
        "            id=fc.id,\n",
        "            response={'result':'ok'},\n",
        "        )]\n",
        "    )\n",
        "\n",
        "    print('>>> ', tool_response)\n",
        "    await session.send(tool_response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TcNu3zUNsI_p"
      },
      "source": [
        "Try running it for a first time:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 103
        },
        "id": "ss9I0MRdHbP2",
        "outputId": "3b00233d-8701-4397-c7b6-f3065f09e353"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "Hello?"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "Hello"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " there! How can I help you today?\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "await run(prompt=\"Hello?\", tools=None, modality = \"TEXT\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z_BFBLLGp-Ye"
      },
      "source": [
        "## Simple function call"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RMq795G6t2hA"
      },
      "source": [
        "The function calling feature of the API Can handle a wide variety of functions. Support in the SDK is still under construction. So keep this simple just send a minimal function definition: Just the function's name.\n",
        "\n",
        "Note that in the live API function calls are independent of the chat turns. The conversation can continue while a function call is being processed."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "8Y00qqZZt5L-"
      },
      "outputs": [],
      "source": [
        "turn_on_the_lights = {'name': 'turn_on_the_lights'}\n",
        "turn_off_the_lights = {'name': 'turn_off_the_lights'}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 189
        },
        "id": "8dCjPmz8nEbv",
        "outputId": "f87f3881-f19c-4657-c6f6-395f5de645b3"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "Turn on the lights"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "``` python\n",
              "print(default_api.turn_on_the_lights())\n",
              "\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            ">>>  function_responses=[FunctionResponse(id='function-call-4840106239621105619', name='turn_on_the_lights', response={'result': 'ok'})]\n"
          ]
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "```\n",
              "{'result': 'ok'}\n",
              "\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "OK"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "prompt = \"Turn on the lights\"\n",
        "\n",
        "tools = [\n",
        "    {'function_declarations': [turn_on_the_lights, turn_off_the_lights]}\n",
        "]\n",
        "\n",
        "await run(prompt, tools=tools, modality = \"TEXT\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eCnCiTbhqE8q"
      },
      "source": [
        "## Code execution"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P4ptRBNY4N8Q"
      },
      "source": [
        "The `code_execution` lets the model write and run python code. Try it on a math problem the model can't solve from memory:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 924
        },
        "id": "k4dURhC-QoSw",
        "outputId": "bd849ecc-1f31-4c10-84ca-6b6497875b2f"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "What is the largest prime palindrome under 100000."
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "Okay"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              ", I understand. You're asking for the largest prime number that is also"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " a palindrome (reads the same forwards and backward) and is less than 1"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "00,000.\n",
              "\n",
              "Here's my plan:\n",
              "\n",
              "1. **Generate Palindromes:** I'll need to create a list of"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " palindromic numbers under 100,000. I'll start from the top, and work my way down since I need the *"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "largest*.\n",
              "2. **Check for Primality:** I'll then test each of these palindromes for primality.\n",
              "3. **Return the Largest Prime:** The largest prime palindrome encountered will be the answer.\n",
              "\n",
              "Let's"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " start by generating and checking the numbers using python.\n",
              "\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "``` python\n",
              "def is_palindrome(n):\n",
              "    return str(n) == str(n)[::-1]\n",
              "\n",
              "def is_prime(n):\n",
              "    if n < 2:\n",
              "        return False\n",
              "    for i in range(2, int(n**0.5) + 1):\n",
              "        if n % i == 0:\n",
              "            return False\n",
              "    return True\n",
              "\n",
              "largest_prime_palindrome = 0\n",
              "for i in range(99999, 1, -1):\n",
              "    if is_palindrome(i):\n",
              "        if is_prime(i):\n",
              "          largest_prime_palindrome = i\n",
              "          break\n",
              "\n",
              "print(f'{largest_prime_palindrome=}')\n",
              "\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "```\n",
              "largest_prime_palindrome=98689\n",
              "\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "Okay"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              ", I have found the largest prime palindrome under 100,00"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "0.\n",
              "\n",
              "The code generated a list of palindromes by checking every number from"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " 99999 downwards. It then tested each of the numbers to see if it was prime. The first prime palindrome found going downwards was 9"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "8689, so that is the largest one.\n",
              "\n",
              "Therefore, the answer is 98689.\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "prompt=\"What is the largest prime palindrome under 100000.\"\n",
        "\n",
        "tools = [\n",
        "    {'code_execution': {}}\n",
        "]\n",
        "\n",
        "await run(prompt, tools=tools, modality='TEXT')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ueeerkpX5F-v"
      },
      "source": [
        "## Compositional Function Calling\n",
        "\n",
        "Compositional function calling refers to the ability to combine user defined functions with the `code_execution` tool. The model will write them into larger blocks of code, and then pause execution while it waits for you to send back responses for each call.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        },
        "id": "XzKyL_Rq5sG3",
        "outputId": "8dd104da-57e9-4bee-8eca-6a8ab1e01d0f"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "Can you turn on the lights wait 10s and then turn them off?"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "``` python\n",
              "import time\n",
              "default_api.turn_on_the_lights()\n",
              "time.sleep(10)\n",
              "default_api.turn_off_the_lights()\n",
              "\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            ">>>  function_responses=[FunctionResponse(id='function-call-9784925699053868540', name='turn_on_the_lights', response={'result': 'ok'})]\n",
            ">>>  function_responses=[FunctionResponse(id='function-call-4771996165652328259', name='turn_off_the_lights', response={'result': 'ok'})]\n"
          ]
        }
      ],
      "source": [
        "prompt=\"Can you turn on the lights wait 10s and then turn them off?\"\n",
        "\n",
        "tools = [\n",
        "    {'code_execution': {}},\n",
        "    {'function_declarations': [turn_on_the_lights, turn_off_the_lights]}\n",
        "]\n",
        "\n",
        "await run(prompt, tools=tools, modality='TEXT')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G78sxDEcqHyO"
      },
      "source": [
        "## Google search"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FW10vRPN6UNp"
      },
      "source": [
        "The `google_search` tool lets the model conduct google searches. For example, try asking it about events that are too recent to be in the training data.\n",
        "\n",
        "The search still executes in `AUDIO` mode, but you don't see the detailed results. So switch to text mode to see the full output:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 660
        },
        "id": "QKvWzROJic60",
        "outputId": "22976e9c-62c5-4d0c-b0e7-b2550942ca45"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "Can you use google search tell me about the largest earthquake in california the week of Dec 5 2024?"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "``` python\n",
              "print(google_search.search(queries=[\"largest earthquake in California week of December 5 2024\", \"California earthquakes week of December 5 2024\"]))\n",
              "\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "```\n",
              "Looking up information on Google Search.\n",
              "\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "The"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " largest earthquake in California during the week of December 5, 202"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "4, was a magnitude 7.0 earthquake that occurred offshore of Cape Mend"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "ocino on December 5th at 10:44 a.m. PT. The epicenter was located approximately 100 km southwest of"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " Ferndale, a city in Humboldt County in Northern California.\n",
              "\n",
              "Here's a breakdown of what is known about the earthquake:\n",
              "\n",
              "*   **Magnitude:**"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " 7.0\n",
              "*   **Date:** December 5, 2024\n",
              "*   **Time:** 10:44 a.m. PT\n",
              "*   **Location:** Offshore of Cape Mendocino"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              ", about 100 km southwest of Ferndale, CA\n",
              "*   **Tsunami Warning:** A tsunami warning was issued for coastal areas from Santa Cruz into Oregon but was canceled shortly after as no significant waves were detected.\n",
              "*"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "   **Aftershocks:** There were numerous aftershocks, including a magnitude 4.7 aftershock that occurred a couple of minutes after the main quake. By Friday morning there were 9 aftershocks of 4.0 magnitude or greater out of nearly 200 aftershocks.\n",
              "*"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "   **Damage:** While there was no widespread damage, some homes and businesses in the region reported minor damage such as items falling off shelves, broken windows, and ruptured water pipes. Some residents reported that the inside of their homes looked like a \"war zone\". Power outages were also reported, affecting over 10,"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "000 people in Humboldt County.\n",
              "*   **Felt Area**: The earthquake was felt as far south as the Bay Area, where residents reported a rolling motion for several seconds.\n",
              "\n",
              "This earthquake was one of the largest to occur in California in recent years and occurred in a seismically active area where three tectonic"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " plates meet.\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "prompt=\"Can you use google search tell me about the largest earthquake in california the week of Dec 5 2024?\"\n",
        "\n",
        "tools = [\n",
        "   {'google_search': {}}\n",
        "]\n",
        "\n",
        "await run(prompt, tools=tools, modality='TEXT')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HM9y5rwfqKfY"
      },
      "source": [
        "## Multi-tool\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qrxAQjYA6vQX"
      },
      "source": [
        "The biggest difference with the new API however is that you're no longer limited to using 1-tool per request. Try combining those tasks from the previous sections:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "QmB_4XPOslyA",
        "outputId": "098edf59-262a-4900-f761-819e68584b84"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "  Hey, I need you to do three things for me.\n",
              "\n",
              "  1. Then compute the largest prime plaindrome under 100000.\n",
              "  2. Then use google search to lookup unformation about the largest earthquake in california the week of Dec 5 2024?\n",
              "  3. Turn on the lights\n",
              "\n",
              "  Thanks!\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "Okay"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              ", I will perform those tasks for you. First, let's find the"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " largest prime palindrome under 100000.\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "``` python\n",
              "def is_palindrome(n):\n",
              "  return str(n) == str(n)[::-1]\n",
              "\n",
              "def is_prime(n):\n",
              "  if n <= 1:\n",
              "    return False\n",
              "  if n <= 3:\n",
              "    return True\n",
              "  if n % 2 == 0 or n % 3 == 0:\n",
              "    return False\n",
              "  i = 5\n",
              "  while i * i <= n:\n",
              "    if n % i == 0 or n % (i + 2) == 0:\n",
              "      return False\n",
              "    i += 6\n",
              "  return True\n",
              "\n",
              "largest_palindrome_prime = 0\n",
              "for i in range(99999, 1, -1):\n",
              "    if is_palindrome(i) and is_prime(i):\n",
              "        largest_palindrome_prime = i\n",
              "        break\n",
              "\n",
              "print(largest_palindrome_prime)\n",
              "\n",
              "\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "```\n",
              "98689\n",
              "\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "Okay"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              ", the largest prime palindrome under 100000 is 9"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "8689.\n",
              "\n",
              "Next, I will search for the largest earthquake in"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " California the week of December 5, 2024.\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "``` python\n",
              "concise_search(\"largest earthquake california week of December 5 2024\", max_num_results=3)\n",
              "\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "```\n",
              "Looking up information on Google Search.\n",
              "\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "Based"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " on the search results, there was a significant earthquake off the coast of Northern California"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " on December 5, 2024. It appears to have been"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " a magnitude 7.0 earthquake near Ferndale, California, with the epicenter offshore. A tsunami warning was issued but later canceled. There were reports of shaking"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              " felt as far away as San Francisco, some minor damage, and some power outages but thankfully, no major injuries or damage were reported. There were several aftersh"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "ocks, including one that initially registered as 5.8 but was downgraded to 4.1.\n",
              "\n",
              "Finally, I will turn on the lights.\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "``` python\n",
              "default_api.turn_on_the_lights()\n",
              "\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------------------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            ">>>  function_responses=[FunctionResponse(id='function-call-7430743998951618653', name='turn_on_the_lights', response={'result': 'ok'})]\n"
          ]
        }
      ],
      "source": [
        "prompt = \"\"\"\\\n",
        "  Hey, I need you to do three things for me.\n",
        "\n",
        "  1. Then compute the largest prime plaindrome under 100000.\n",
        "  2. Then use google search to lookup unformation about the largest earthquake in california the week of Dec 5 2024?\n",
        "  3. Turn on the lights\n",
        "\n",
        "  Thanks!\n",
        "  \"\"\"\n",
        "\n",
        "tools = [\n",
        "    {'google_search': {}},\n",
        "    {'code_execution': {}},\n",
        "    {'function_declarations': [turn_on_the_lights, turn_off_the_lights]}\n",
        "]\n",
        "\n",
        "await run(prompt, tools=tools, modality=\"TEXT\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y0OhM95KkMzl"
      },
      "source": [
        "## Next Steps\n",
        "\n",
        "- For more information about the SDK see the [SDK docs](https://googleapis.github.io/python-genai/)\n",
        "- This tutorial uses the high level SDK, if you're interested in the lower-level details, try the [Websocket version of this tutorial](../gemini-2/websocket/search_tool.ipynb)\n",
        "- This tutorial only covers _basic_ usage of these tools for deeper (and more fun) example see the [Search tool tutorial](../gemini-2/search_tool.ipynb)\n",
        "\n",
        "Or check the other Gemini 2.0 capabilities from the [Cookbook](https://github.com/google-gemini/cookbook/blob/main/gemini-2/), in particular this other [multi-tool](../gemini-2/plotting_and_mapping.ipynb) example and the one about Gemini [spatial capabilities](../gemini-2/spatial_understanding.ipynb)."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
